FIG. 1 illustrates a computer network 105 connecting a client 110 and a conventional server 130. The computer network 105 may be a public network, such as the Internet, or a private network, such as a LAN, MAN or WAN. The client 10 is a computer, such as a personal computer, hand held Internet terminal, Internet enabled television terminal or Internet enabled cellular telephone. The server 130 may host a website on the Internet, for example. The server 130 comprises a server application 115 redundantly provisioned in several (say, M) computers. Each server application 115-1 through 115-M is stateless (i.e., operation does not depend upon the historical state of the server application 115-1 through 115-M) and identical. A load balancer 120 connects between the computer network 105 and the server applications 115-1 through 115-M. The load balancer 120 routes connections to the server applications 115 in an even handed way. When the client 110 requests a connection to the server application 115 (in the aggregate, without distinction to a specific one of the redundant instances of the server application 115-1 through 115-M), the load balancer 120 receives the request and forwards the request to the least loaded server application 115. The server applications optionally connect to a separate centralized database 125, as shown. Alternatively, each server application II 5 can include its own copy of the database 125. Together the server applications 115, the load balancer 120 and the database 125 appear to the client 110 as a single entity—the server 130—on the computer network 105.
The server 130 illustrated in FIG. 1 is ill-equipped to deal with surges in demand. Surges can arise for a myriad of reasons. For example, when the computer network 105 is highly dynamic, as is the Internet, the pattern of usage of a website server is highly variable and subject to change quickly. Frequently demand for a website surges, as its popularity increases dramatically. The arrival and duration of surges are difficult to predict. When a surge occurs, the level of service provided to all users can drop significantly, causing discontent among the users of the website. This surge-induced drop-off in service can counteract the website's initial popularity, causing the initial popularity to be merely temporary, as users refuse to tolerate slow responses or denied connection requests. If the surge is attributable to investments in advertising or other good fortune, then the drop-off in service can limit any gains expected to result from the, investment or good fortune.
In the framework of FIG. 1, there are two approaches for dealing with surges. The first approach is the over-provision of server applications 115. That is, a sufficient number of server applications 115 are provided to handle peak demand. A disadvantage of this approach is that it is difficult to predict precisely how many server applications 115 are needed to handle future surges. Another disadvantage of this approach is that some of the server applications 115 sit idle a great majority of the time. Over-provision is therefore difficult to plan effectively and extremely inefficient, even if planned effectively.
The second approach is a manual reaction to a surge. According to this approach, a human operator notices that demand exceeds server capacity and manually purchases additional server applications 115. This approach is severely limited in several respects. First, there is a significant time lag between the surge of demand and the operation of the new server applications. Second, once a new server application has been purchased, it will remain allocated and idle during periods of low demand. Thus, manual reaction is a slow and inefficient way of dealing with surges.